Block-oriented control system on high speed ethernet

ABSTRACT

A distributed control system architecture (HSE) provides an open, interoperable solution optimized for integration of distributed control systems and other control devices in a high performance backbone, provides an open, interoperable solution that provides system time synchronization suitable for distributed control applications operable over a high performance backbone, and provides an open, interoperable solution that provides a fault tolerant high performance backbone as well as fault tolerant devices that are connected to the backbone. The distributed control system architecture comprises a High speed Ethernet Field Device Access (HSE FDA) Agent, which maps services of a distributed control system, e.g., a fieldbus System, to and from a standard, commercial off-the-shelf (COTS) Ethernet/Internet component. The distributed control system architecture also comprises a High speed Ethernet System Management Kernel (HSE SMK) that operates to keep a local time, and keeps the difference between the local time and a system time provided by a time server within a value specified by the time sync class. The local time is used to time stamp events so that event messages from devices may be correlated across the system. The distributed control system architecture further comprises a High speed Ethernet Local Area Network Redundancy Entity (HSE LRE) that provides redundancy transparent to the applications running on the system. The HSE LRE of each device periodically transmits a diagnostic message representing its view of the network to the other Devices on the system. Each device uses the diagnostic messages to maintain a Network Status Table (NST), which is used for fault detection and selection from a redundant pair of resources.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 09/598,697, filed Jun. 21, 2000; which claims the benefit of U.S.Provisional Application No. 60/139,814, filed Jun. 21, 1999; and whichis a continuation-in-part (CIP) application of U.S. application Ser. No.08/916,178, filed Aug. 21, 1997; which claims the benefit of U.S.Provisional Application No. 60/024,346, filed Aug. 23, 1996; all ofwhich are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present invention relates to control system architecture. Moreparticularly, the present invention relates to an open, interoperabledistributed control system in a high performance network environment.

BACKGROUND OF THE INVENTION

Automatic control systems are critical to all sectors of industry suchas process control, discrete control, batch control (process anddiscrete combined), machine tool control, motion control, and robotics.One of the strongest needs in modern control systems is development anduse of “open” and “interoperable” systems. Open, interoperable systemsallow control devices made by different manufacturers to communicate andwork together in the same system without the need for customprogramming. “Fieldbus” is the common term used to describe these typesof control systems.

The movement toward open, interoperable fieldbus systems is driven bydevice manufacturers and end users. Manufacturers want open,interoperable systems because it allows them to sell their products tomore end users while reducing development costs. End users want open,interoperable systems so that they can select the best control devicesfor their system regardless of the device manufacturer.

There has also been a trend toward distribution of control functionsinto intelligent devices. In centralized control systems, a centralcontroller performs all the control functions.

In distributed control systems, more than one control device operatingin the system takes an active role in the control functions. Althoughboth centralized and decentralized systems use a communication network,decentralized systems reduce overall system costs by reducing oreliminating the centralized controller functions between the controldevices and the human-machine interface.

In order for distributed control systems to be truly open andinteroperable, both the communications system and the user layer (abovethe communication system layers) must be specified and made open. One ofthe truly open and interoperable distributed systems is the fieldbussystem provided by the Fieldbus Foundation. The FOUNDATION™ Fieldbususer layer is described, e.g., in U.S. patent application Ser. No.08/916,178 (hereafter the “'178” application) filed Aug. 21, 1997,entitled “BLOCK-ORIENTED CONTROL SYSTEM”, and assigned to the assigneeof the present application.

The lower speed 31.25 kilobits per second fieldbus (H1) used by theFOUNDATION™ fieldbus is described in part by InternationalElectrotechnical Committee (IEC) Standard IEC 61158, the entirety ofwhich is hereby incorporated by reference herein.

While the FOUNDATION™ fieldbus provides the open and interoperablesolution for the H1 control capability, there is a great need to providean open and interoperable solution for distributed control on a veryhigh performance communication system typically called a fieldbus“backbone” network. The backbone network aggregates information from thelower speed control devices, e.g., the H1 and other control devices,which is used in supervisory and advanced control applications. Thebackbone is also needed for integration of control information into theenterprise's Management Information Systems (MIS).

One of the widely accepted standards for high performance communicationssignaling is Ethernet. Invented by Xerox in the 1970's, Ethernet hasprogressed from an initial speed of 10 Megabits per second, to 100Megabits per second, to 1 Gigabit per second and beyond. Ethernetsignaling is specified in an Institute of Electrical and ElectronicsEngineers (IEEE) standard (IEEE 802.3). Ethernet signaling is theunderlying technology used by the Internet. The Internet protocols arespecified by the Internet Engineering Task Force (IETF) and are issuedas Request for Comment (RFC) specifications.

Although Ethernet/Internet technology provides the basic services for ahigh performance fieldbus backbone, it does not provide for all of thefunctions needed for use in distributed control systems. In particular,IEEE and IETF do not have suitable open and interoperable solutions forintegration of distributed control systems (e.g., the H1 subsystem),system time synchronization, and fault tolerance.

The method of transferring information from lower speed fieldbuses tothe Ethernet used by organizations such as Open DeviceNet™ VendorAssociation, Inc., (“EtherNet/IP,”) and PROFIBUS International,(“PROFINet”) are not suitable for use in the high performanceenvironment because they encapsulate the lower speed protocol packets inan Ethernet frame. This method, known as “tunneling,” is common incentralized control systems, but is inadequate for high performancedistributed control systems. Although simpler to specify, tunnelingwould require too many Transport Control Protocol (TCP) connections withthe resulting interrupt processing and memory overhead on the devicesconnected to the fieldbus backbone. In addition tunneling wastes much ofthe Ethernet bandwidth because the lower speed protocol packets (e.g.,the H1 packets) are small and in many cases the Ethernet packet overheadwould be bigger than a lower speed protocol packet.

Devices connected to the Ethernet must have a common sense of systemtime for time stamp and function block scheduling (control) purposes.For high performance distributed control, system time often needs to beaccurate to within less than 1 millisecond. Heretofore, there is noknown solution that provides this accuracy using the Commercial Off TheShelf (COTS) Ethernet equipment.

Fault tolerance of the Ethernet communication media and devicesconnected to the Ethernet is required for high performance distributedcontrol applications. There is no known solution that provides therequired fault tolerance using standard COTS Ethernet equipment. All ofthe prior attempts in providing the required fault tolerance requirespecial Ethernet/Internet electronic hardware and/or software, and/or anon-standard “redundancy manager” device to be added to the Ethernet.

Thus, what is needed is an open, interoperable solution optimized forintegration of distributed control systems and other control devices ina high performance fieldbus backbone.

What is also needed is an open, interoperable solution that providessystem time synchronization suitable for distributed controlapplications operable over a high performance fieldbus backbone.

What is also needed is an open, interoperable solution that provides afault tolerant high performance fieldbus backbone as well as faulttolerant devices that are connected to the fieldbus backbone.

SUMMARY OF THE INVENTION

The present invention overcomes the shortcomings described above andprovides a new and improved distributed control system, which operateson a high performance backbone, e.g., the standard COTS Ethernet andInternet technology. The embodiments of the present invention arecollectively referred to herein as the “High Speed Ethernet” (HSE). HSEincludes the features of the distributed control system described by the'178 application and FOUNDATION™ fieldbus specifications (which arelisted in Appendix A as the Reference Set 1), and further includes threenew protocols described in the supporting specifications thereof, whichare listed in Appendix A as the Reference Set 2. In particular, the newprotocols are referred to herein as: the HSE Field Device Access (FDA)Agent, the HSE System Management Kernel (SMK), and the HSE Local AreaNetwork Redundancy Entity (LRE).

The HSE FDA Agent allows System Management (SM) and Fieldbus MessageSpecification (FMS) services used by the H1 devices to be conveyed overthe Ethernet using standard Internet User Data Protocol (UDP) andTransport Control Protocol (TCP). This allows HSE Devices on theEthernet to communicate to H1 devices that are connected via a “HSELinking Device.” The HSE FDA Agent is also used by the local FunctionBlock Application Process (FBAP) in a HSE Device or HSE Linking Device.Thus, the HSE FDA Agent enables remote applications to access HSEDevices and/or H1 devices through a common interface.

The HSE SMK ensures that system level functions in each device arecoordinated. These functions include system time, addition and removalof devices from the network, and function block scheduling. HSE SMK useslocal clock that operates to keep a local time, and keeps the differencebetween the local time and a system time provided by a time serverwithin a value specified by the time sync class (See Reference Set 1 ofAppendix A herein). The local time is used to time stamp events so thatevent messages from devices may be correlated across the system. Localtime is also used to schedule the execution of the local functionblocks.

HSE fault tolerance is achieved by operational transparency i.e., theredundancy operations are not visible to the HSE applications. This isnecessary because HSE applications are required to coexist with standardMIS applications. The HSE LRE coordinates the redundancy function. EachHSE Device periodically transmits a diagnostic message representing itsview of the network to the other HSE Devices on its Ethernet interfaces(commonly called Ethernet “Ports”). Each device uses the diagnosticmessages to maintain a Network Status Table (NST), which is used forfault detection and Ethernet transmission port selection. There is nocentral “Redundancy Manager”. Instead, each device determines how itshould behave in response to faults it detects.

BRIEF DESCRIPTION OF THE DRAWINGS

Features and advantages of the present invention will become apparent tothose skilled in the art from the following description with referenceto the drawings, in which:

FIG. 1 is a block diagram showing an exemplary embodiment of a highperformance distributed control system in accordance with the principalsof the present invention;

FIG. 2 is a block diagram showing an exemplary embodiment of devicesystem architecture of a high performance distributed control system inaccordance with the principles of the present invention;

FIG. 3 is a block diagram showing an exemplary embodiment of thestructure of the High Speed Ethernet Management Information Base of thedevice system architecture shown in FIG. 2;

FIG. 4 is a block diagram showing an exemplary embodiment of the devicesystem architecture shown in FIG. 2, showing the various localinterfaces of the High Speed Ethernet Field Device Access agent;

FIG. 5 is a block diagram showing an exemplary embodiment of therelevant portions of the high performance distributed control systeminvolved in time synchronization process in accordance with theprinciples of the present invention;

FIG. 6 is a flow diagram illustrative of an exemplary embodiment of theprocess of time synchronization in accordance with an embodiment of theprinciples of the present invention;

FIG. 7A is a timing diagram illustrative of a starting time offsetbefore the time synchronization process in accordance with an embodimentof the principles of the present invention;

FIG. 7B is a timing diagram illustrative of a starting time offset afterthe time synchronization process in accordance with an embodiment of theprinciples of the present invention; and

FIG. 8 is a block diagram showing an exemplary embodiment of theredundant topology of a high performance distributed control system inaccordance with the principles of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

For simplicity and illustrative purposes, the principles of the presentinvention are described by referring mainly to exemplary embodiments,particularly, with specific exemplary implementations of distributedcontrol system in an Ethernet network. However, one of ordinary skill inthe art would readily recognize that the same principles are equallyapplicable to, and can be implemented in, other implementations anddesigns using any other high speed networks, and that any such variationwould be within such modifications that do not depart from the truespirit and scope of the present invention.

A: HSE Distributed Control System Overview

Referring to FIG. 1, an example of a high performance control system 100is shown where standard COTS Ethernet equipment 130 is used tointerconnect HSE Linking Devices 110 and HSE Devices 120 to an EthernetNetwork 140. The HSE Linking Devices 110 in turn connect to H1 Devices170 using H1 Networks 150. Other types of equipment such as PersonalComputer (PC) 160 may also be connected to the Ethernet Network 140.

The actual Ethernet network topology and COTS Ethernet equipmentconfiguration will depend on the particular application needs. Howeverany Ethernet network topology or configuration using standard COTSEthernet equipment other than the exemplary topology shown in FIG. 1 maybe used, and such variations would be within such modifications that donot depart from the true spirit and scope of the present invention.

A.1: HSE System Architecture

The HSE system architecture in accordance with an embodiment of theprinciples of the present invention is shown in FIG. 2. The HSE systemarchitecture is designed to meet the functional needs of the highperformance distributed manufacturing and process control environments,e.g., in a high speed Ethernet network. It permits distributedautomation systems to be constructed from various control andmeasurement devices manufactured by different vendors. The HSE systemarchitecture is described by architecture components that have beenadapted to the specifics of both the H1 and HSE environments.

The various protocols and standards referenced in the followingdisclosure are described in detail in the manuals and specificationslisted in Appendix A herein, which are available from the FieldbusFoundation, a not-for-profit organization headquartered in Austin, Tex.,and the respective current versions as of the filing date of the presentinvention of all of which are hereby incorporated by reference in theirentirety herein. Each of the architecture components of the HSE systemarchitecture (shown in FIG. 2) will now be described in more detail.

A.2: Function Block Application Process Virtual Field Device (FBAP VFD)

Application Process (AP) is a term defined by the InternationalStandards Organization (ISO) Open Systems Interconnect (OSI) ReferenceModel (RM), ISO 7498, to describe the portion of a distributedapplication that is resident in a single device. The term is used in thefollowing description to refer to the entities within a device thatperforms a related set of functions, such as function block processing,network management, and system management.

Virtual Field Device (VFD) is a term defined by the Fieldbus Foundation(See Fieldbus Message Specification FF-870 listed in Reference Set 1 inAppendix A herein). A VFD makes the parameters of an AP visible to acommunication network.

In accordance with the principles of the present invention, the HSEsystem architecture (shown in FIG. 2) supports the Function BlockApplication Process Virtual Field Device (FBAP VFD) 260. The FBAP VFD260 provides a common means for defining inputs, outputs, algorithms,control variables, and behavior of the automation system. There may bemultiple FBAP VFDs 260, e.g., n FBAP VFDs as shown, in a device in orderto satisfy the particular needs an application. The FBAP VFD may or maynot be present in a HSE Device or HSE Linking Device. If the HSE FBAPVFD is present, the device is sometimes also referred to as a “HSE FieldDevice.” In the following description, however, the FBAP VFD is to beassumed to be present in the HSE Device and HSE Linking Device, even ifthe term “HSE Field Device” is not used.

A standard set of function block classes and parameters are defined bythe Fieldbus Foundation, e.g., in one or more of the specificationslisted in Appendix A herein. Manufacturers of control devices may appendtheir own parameters to the standard set of parameters to accommodateadditional function block definitions as new requirements arediscovered, and as technology advances. A more detailed description ofthe function block classes and parameters may be found, e.g., inFunction Block Application Process-Part 1 Specification FF-890 listed inReference Set 1 of Appendix A herein.

A.3: H1 Interface

Each H1 Network 150 attached to a HSE Linking Device 110 (shown inFIG. 1) requires a H1 Interface 240. The Bridge 250 is used to convey H1Network messages directly between other H1 Interfaces 240 within thesame HSE Linking Device 110 (shown in FIG. 1). A HSE Linking Device maycomprise, e.g., a HSE Device 120 (shown in FIG. 1) that includes atleast one H1 Interface 240.

A more detailed description of a H1 Interface may be found in theFieldbus Message Specification FF-870, Fieldbus Access SublayerSpecification FF-821, Data Link Services and Data Link ProtocolSpecifications FF-821, 822, and Data Link Protocol Specification forBridge Operation Addendum FF-806, all of which are listed in theReference Set 1 of Appendix A herein.

A.4: Ethernet/Internet Stack

The HSE system architecture uses a standard COTS Ethernet/Internet(“stack”) 280 for communication with other devices on the EthernetNetwork 140. The Ethernet/Internet stack used by HSE consists ofDistributed Hose Control Protocol (DHCP) 285, Simple Network TimeProtocol (SNTP) 286, and Simple Network Management Protocol (SNMP) 287,which in turn use Transport Control Protocol (TCP) 283 and User DataProtocol (UDP) 284 services.

TCP 283 and UDP 284 in turn use the standard Internet Protocol (IP) 282services, which uses the standard IEEE Ethernet 802.3 Media AccessControl (MAC) and Physical (PHY) Layers 281. The PHY layer in 281connects to one or more Ethernet Networks 140.

The Internet DHCP, SNTP, SNMP, TCP, UDP and IP protocols are specifiedby the Internet Engineering Task Force (IETF) Request For Comment (RFC)specifications. The IETF RFCs are listed in Appendix B herein, which arehereby incorporated by reference herein in their entireties. AnInstitute of Electrical and Electronics Engineers (IEEE) standard (IEEE802.3), the entirety of which is hereby incorporated by referenceherein, describe the Ethernet MAC and PHY layers. The specific use ofeach layer and protocol are detailed in the Ethernet PresenceSpecification FF-586 listed in Reference Set 2 of Appendix A herein.

By preserving the standard use of the Ethernet/Internet stack, the HSEsystem architecture ensures interoperability among the different stackmanufacturers.

A.5: HSE Management Agent

Again referring to FIG. 2, in general, the HSE Management Agent 270 usesDHCP 285 for acquiring an IP address for the device, SNTP 286 formaintaining time synchronization with a time server, and SNMP 287 formanaging the TCP, UDP, and IP protocol layers. HSE Management Agent useof DHCP, SNTP and SNMP is routine and complies with standard practicesknown to those familiar with the Internet protocols, e.g., according toIEEE 802.3.

The HSE Management Agent uses SNMP 287 for managing the Internet layerprotocols. Specifically, the HSE Management Agent 270 provides EthernetNetwork access to the standard Management Information Base-II (MIB II)as defined by SNMPv2 in RFC 1213 and RFC 1643 (see Appendix B), and asdefined also by Ethernet Presence FF-586 listed in Reference Set 2 ofAppendix A herein.

In accordance with an embodiment of the present invention, in order tocomply with the ISO standards, the HSE Management Information Base (HSEMIB) 271 comprises of a standard part, which is the second version ofMIB-11 as defined in RFC 1213 and a HSE specific part (which is definedunder the private enterprises level). For convenience in understanding,the detailed structure of the HSE MIB 271 is shown in FIG. 3. Thestandardized structure of the HSE MIB 271 provides a profile allowinginteroperability, making the device appear as a well-behaved node.

B: HSE Core

Referring again to FIG. 2, the HSE core portion 200 of the HSE systemarchitecture identifies the new HSE capability in accordance with theprinciples of the present invention. The HSE core 200 provides theessential capabilities and integration needed to realize the highperformance distributed control using HSE Devices, HSE Linking Devicesand standard COTS Ethernet equipment.

B.1: Network Management Agent Virtual Field Device

The HSE System Architecture includes a Network Management Agent VFD (NMAVFD) 210 for each HSE Device and each HSE Linking Device. The NMA VFDprovides means for configuring, controlling and monitoring HSE Deviceand HSE Linking Device operation from the network.

Management information is contained in the Network ManagementInformation Base (NMIB) 213 and the System Management Information Base(SMIB) 212. Using the configuration management capabilities of the NMAVFD, parameters are set in the NMIB and SMIB to support data exchangeswith other devices in the system. This process involves defining thetransfers between devices and then selecting the desired communicationscharacteristics to support the transfers.

The NMA VFD can also be configured to collect performance and faultrelated information for selected transfers. This information isaccessible during run-time, making it possible to view and analyze thebehavior of device communications. If a problem is detected, performanceis to be optimized, or device communications are to be changed, thenreconfiguration can be performed dynamically while the device is tilloperating.

NMA VFD parameters and behavior are further defined in the HSE NetworkManagement Specification FF-803 listed in the Reference Set 2 ofAppendix A herein.

B.2: HSE Field Device Access Agent

The HSE Field Device Access (FDA) Agent will now be described withReferences to FIG. 4, which is the same figure as FIG. 2 except thatLocal Interactions (291-299) for the HSE Field Device Access (FDA) Agent290 are shown. The operation of the HSE FDA Agent will now be describedin terms of these local interactions.

One of the main functions of the HSE FDA Agent 290 is to map servicesalready defined for FOUNDATION™ fieldbus System Management (SM) (SeeFF-880 listed in the Reference Set 1 of Appendix A herein) and FieldbusMessage Specification (FMS) (See FF-870 listed in the Reference Set 1 ofAppendix A herein) to an from the standard, COTS Ethernet/Internet 280component.

Generally, the HSE FDA Agent 290 emulates the mapping defined by theFOUNDATION™ fieldbus Fieldbus Access Sublayer specification (See FF-875listed in the Reference Set 1 of Appendix A herein). The HSE FDA Agent290 provides the common interface that enables remote applications toaccess devices of any type on both the H1 Networks 150 and the HSENetwork 140.

Thus the HSE FDA Agent 290 in accordance with the principles of thepresent invention allows systems to be constructed where the control isdistributed in into various HSE Devices and/or H1 Devices, and anycombinations thereof, as needed by the particular and user application.

B.2.1: HSE FDA Agent Local Interfaces

B.2.1.(a): Local Interface 291: TCP—The TCP local interface 291 allowsthe HSE FDA Agent 290 to send and/or receive FMS messages using TCP 283.TCP 283 provides interfaces modeled as sockets through which the HSE FDAAgent 290 submits a buffer that contains one or more messages.

B.2.1.(b): Local Interface 292: UDP—The UDP local interface 292 allowsthe HSE FDA Agent 290 to send and/or receive SM messages and certain FMSmessages using UDP 284. UDP 284 provides interfaces modeled as socketsthrough which the HSE FDA Agent 290 submits a buffer that contains oneor more messages.

B.2.1.(c): Local Interface 293: HSE NMIB—The HSE FDA Agent 290 providesa local interface to the HSE NMIB 213 in NMA VFD 210. The HSE FDA Agentis capable of providing configuration and read-only access to NMA VFD210 via the HSE NMIB Local Interface 293.

B.2.1.(d): Local Interface 294: HSE SMIB—The HSE FDA Agent 290 providesa local interface to the HSE SMIB 212 in NMA VFD 210. The HSE FDA Agent290 is capable of providing configuration and read-only access to NMAVFD 210 via the HSE SMIB Local Interface 294.

B.2.1.(e): Local Interface 295: HSE SMK—The HSE FDA Agent 290 conveysHSE SM services to and from the HSE SMK 220 through the HSE SMK localinterface 295. In accordance with an embodiment of the presentinvention, in a HSE Linking Device, the HSE SMK 220 communicates locallywith each of the H1 interfaces 240, and does not use the HSE FDA Agent290.

B.2.1.(f): Local Interface 296: HSE LRE—The HSE FDA Agent 290 maintainsa local interface with the HSE LAN Redundancy Entity (HSE LRE) 230 ofthe device through the HSE LRE local interface 296. Use of the HSE LRElocal interface 296 will be described in more detail later.

B.2.1.(g): Local Interface 297: H1 Interface—Only HSE FDA Agents 290 ofa HSE Linking Device interact with the H1 Interface(s) 240 to access H1Networks 150. The H1 local interface provides the HSE FDA Agent with FMSand SM access through the HSE SMK 220.

The HSE FDA Agent forwards FMS requests and responses received form theTCP Interaction 291 and UDP Interaction 292 to H1 Network 150 Throughthe H1 Interface(s) 240. The HSE FDA Agent also forwards H1 requests andresponses received from a H1 Network through the H1 InterfaceInteraction 297 to the Ethernet Network 140 using TCP Interaction 291and UDP Interaction 292.

Thus, the HSE FDA Agent 290 interacts with the services in the H1Network in the same manner as any other application program wouldnormally interact with the H1 network.

B.2.1.(h): Local Interface 298: FBAP VFD—The HSE FDA Agent 290 uses theFBAP VFD local interface 298 to access the FBAP VFD 260. Both FMS and SMmessages are communicated using the FBAP VFD local interface 298.

B.2.1.(i): Local Interface 299: HSE Management Agent—The HSE FDA Agent290 maintains the HSE Management Agent local interface 299 with the HSEManagement Agent 270 to access certain Quality of Service parametersassociated with its UDP/TCP connections. The use of these parameters bythe HSE FDA Agent 290 is local to the specific UDP/TCP implementation.

B.2.2.: HSE FDA Agent Operation

Again Referring to FIG. 4, during the configuration of the system, HSESMK 220 uses the local interface 295 for adding HSE and/or H1 devicesto, and deleting the same from, the distributed system. An exchange ofSM messages is used to identify new (or to be deleted) HSE and/or H1Devices in the system.

For example, after a new HSE Device receives an Internet Protocol (IP)address, the new HSE Device periodically announces its presence on theEthernet network 140. HSE Linking Devices also announce changes detectedon their H1 Network 150. In a similar way, HSE SMK uses the localinterface 295 to determine the location of the function block “tags”that might exist in the HSE Devices and/or H1 Devices.

During operation of the system, the data acquisition, display andsupervisory control functions, which are typically executing on aPersonal Computer (PC) connected to the Ethernet Network 140, will needto access the data in a HSE Device, a HSE Linking Device and/or H1devices connected to the H1 Networks 150. The data access is typicallyperformed using the “Client/Server” and/or the “Publisher/Subscriber”messages. These data access methods are well known to those familiarwith Fieldbus messaging.

For Client/Server and Publisher/Subscriber messages originating orterminating in the HSE Device and/or HSE Linking Device, the HSE FDAAgent 290 sends and receives the Ethernet Network 140 messages on thelocal interface 291, provides the appropriate mapping to FMS services aspreviously described above, and uses local interfaces 293, 294, 296, 298and 299 to access the HSE NMIB 213, HSE SMIB 212, HSE LRE 230, FBAPVFD(s) 260 and the HSE Management Agent 270, respectively. HSE SMK 220is not accessed because it has its own SM messages as previouslydescribed.

For Client/Server, Publisher/Subscriber and/or SM messages originatingor terminating in the H1 Network 150, the HSE FDA Agent 290 uses localinterface 297 to send and/or receive messages from H1 Interface(s) 240.

If the messages from the H1 network 150 are to/from the Ethernet Network140, and are Client/Server or Publisher/Subscriber messages, HSE FDAAgent 290 uses the FMS Mapping and local interface 291. If the H1messages to/from the Ethernet Network 140 are SM messages, the HSE FDAAgent uses the SM mapping and local interface 292.

If the messages to/from H1 Network 150 are to/from the HSE LinkingDevice, and are Client/Server or Publisher/Subscriber messages, HSE FDAAgent will use FMS mapping and the appropriate local interface (exceptthe local interfaces 291 and 292).

If the messages to/from H1 Network 150 are to/from the HSE LinkingDevice, and are SM messages, HSE FDA Agent will use SM mapping and theappropriate local interface (except the local interfaces 291 and 292).

B.3: HSE System Management Kernel

Referring again to FIG. 2, the HSE system architecture includes a HSESystem Management Kernel (SMK) 220 for each HSE device and/or each HSElinking device. The HSE SMK 220 maintains information and a level ofcoordination that provides an integrated network environment for theexecution and interoperation of the FBAP VFD 260.

As previously discussed, HSE SMK 220 provides for routine configurationof certain basic system information prior to device operation. Forexample HSE SMK startup takes a device through a set of predefinedphases for this purpose. During this procedure a system configurationdevice recognizes the presence of the device on the network andconfigures basic information into the HSE SMIB 212. Once the devicereceives its basic configuration information, its HSE SMK brings it toan operational state without affecting the operation of other devices onthe network. It also enables the HSE FDA Agent 290 for use by otherfunctions in the device.

B.3.1.: HSE SMK System Time Synchronization

Now referring to FIG. 5, the HSE Management Agent 270 in HSE LinkingDevice 110 uses SNTP 286 to interact with remote SNTP Server 510 in TimeMaster 500 to synchronize System Time 501′ in HSE MIB 271′ with SystemTime 501 in the Time Master 500. When System Time 501′ is synchronizedwith System Time 501, Sync Flag (F) 510 in the HSE MIB is set to true bythe standard SNTP protocol. The Time Master and the HSE Linking Deviceare interconnected using standard, COTS Ethernet equipment 130. Thissynchronization protocol is defined in IETF RFC 2030.

At any moment, Local Time 502 in HSE SMIB 212 may or may not besynchronized with System Time 501′. In order to coordinate execution offunction blocks in a distributed system, and to provide proper timestamping of function block alarms, Local Time 502 must be Synchronizedwith System Time 501′.

All of the function blocks are synchronized with Start of Macrocycle,“To” 520 in HSE SMIB 212. Each HSE Linking Device and HSE Device in thesystem has the same value for To. A function block is executed when HSESMK 220 locally issues a Function Block (FB) Start 221 message for theblock. Each FB Start message is generated based on an offset from To.

At the start of the Macrocycle, To, and the offset for each block isbased on Local Time 502. Therefore each device must adjust their LocalTime 502 to equal the System Time 501′ for the system to functionproperly. However, because each device has a hardware clock oscillatorthat is not perfect, Local Time 502 will eventually drift out ofsynchronization with System Time 501′.

FIG. 6 shows the process of correcting for the drift in accordance withan embodiment of the present invention. In particular, when a macrocycleends in step 601, the HSE SMK 220 will test the Sync Flag 510 in HSE MIB271′ in step 602. If F 510 is not true, the process ends in step 606.

If, on the other hand, it is determined in step 602 above that F 510 istrue, HSE SMK 220 computes the offset between Local Time 502 and SystemTime 501′ in step 603, and sets the Local Time 502 to equal the SystemTime 501′ within a value specified in a desired time sync class (SeeReference Set 1 of Appendix A herein) in step 604.

Once the Local Time 502 is synchronized, in step 605, the start time(To) 520 (shown in FIG. 5) is aligned with start time of other devices.

The start time alignment will now be described with references to FIGS.7A and 7B. FIG. 7A shows the macrocycle offset of a device, e.g., deviceN, before the time synchronization, in which the offset 720 representsthe error that must be corrected in the HSE Device N. As shown, the HSEDevice N now has the correct Local Time, but the start time (To) 520′,of System Macrocycle 700′ is not aligned with other devices in thedistributed system.

FIG. 7B shows the macrocycle offset of a device, e.g., device N, afterthe time synchronization. The HSE SMK 220 of the Device N delays thestart time (To) 520′ of the System Macrocycle 700′ by Offset 720 so thatthe System Macrocycle begins at the same time (To) 520 as, e.g., SystemMacrocycle 700 in HSE Device 1. HSE Device N System Macrocycle is nowsynchronized with the System Time, and the synchronization process endsat step 606 (shown in FIG. 6).

B.4: Local Area Network Redundancy Entity

Referring to FIG. 4, each HSE Device and HSE Linking Device has a HSELocal Area Network (LAN) Redundancy Entity (HSE LRE) 230. The HSE LREprovides fault tolerance to single failure through the use ofredundancy.

HSE LRE periodically sends and receives Redundancy Diagnostic Messagesover local interface 296. HSE FDA Agent 290 maps the Diagnostic messageson local interfaces 291 and 292 (See HSE Redundancy Specification FF-593listed in the Reference Set 2 of Appendix A herein for the RedundancyDiagnostic Message Formats.)

The Redundancy Diagnostic Messages are sent concurrently on EthernetNetwork 140 and Ethernet Network 140′. Each device receives theRedundancy Diagnostic Messages on Ethernet Network 140 and EthernetNetwork 140′ and constructs a local Network Status Table (NST) 231. TheNST provides detailed status on the condition of every HSE deviceconnected to Ethernet Network 140 and Ethernet Network 140′. The HSE LRE230 controls which Ethernet Network 140 or 140′ the HSE Device will usefor message transmission.

With this method, all of the network transmission and device switchoverdecisions are distributed into the HSE Devices and the system usesstandard, COTS Ethernet equipment.

FIG. 8 illustrates the general topology supported by the redundancyaspect of the present invention. The topology shown is only an example,showing one of many possible topologies. Any topology may be used aslong as behavior of the equipment providing Ethernet Networks 140 and140′ is standard, COTS Ethernet equipment.

HSE redundancy supports both Ethernet Network redundancy and HSE LinkingDevice redundancy.

B.4.1.: Ethernet Network Redundancy

Referring to FIG. 8, HSE Devices 120′ and HSE Linking Device Pairs 110′have interfaces to both Ethernet Network 140 and Ethernet Network 140′.In this example, Ethernet Network 140 is provided by COTS Ethernetequipment 130 and Ethernet Network 140′ provided by COTS Ethernetequipment 130 and Ethernet Network 140′ is provided by COTS Ethernetequipment 130′. A single failure of any one of the Ethernet Networks orone of the Ethernet interfaces on a HSE device would cause the HSE LREpreviously described to force communications on the remaining functionalnetwork.

B.4.2.: HSE Linking Device Redundancy

The HSE LRE 230 supports HSE Linking Device redundancy. Redundant HSELinking Device Pair 160 comprises primary HSE Linking Device 110, andstandby HSE Linking Device 110′. H1 Devices 170 are connected by H1Networks 150 to the Redundant HSE Linking Device Pair 160. If primaryHSE Linking Device 110 fails, standby HSE Linking Device 110′ willassume control. A HSE device 120′ may be made redundant in the samemanner as the HSE linking device 110, except in a HSE device H1interface(s) is (are) not present.

The present invention provides the necessary diagnostic message formatto allow an open and interoperable switch-over of the redundant highspeed Ethernet networks and/or the redundant HSE linking devices (or HSEdevices).

The redundancy method for backup of each H1 Network is described in the'178 application, and by the specifications listed in Reference Set 1 ofAppendix A herein.

As can be appreciated, the distributed control system architecture inthe foregoing description provides an open, interoperable solutionoptimized for integration of distributed control systems and othercontrol devices in a high performance backbone, provides an open,interoperable solution that provides system time synchronizationsuitable for distributed control applications operable over a highperformance backbone, and provides an open, interoperable solution thatprovides a fault tolerant high performance backbone as well as faulttolerant devices that are connected to the backbone.

The preferred embodiments set forth above are to illustrate theinvention and are not intended to limit the present invention.Additional embodiments and advantages within the scope of the claimedinvention will be apparent to one of ordinary skill in the art.

Moreover, while the invention has been described with reference to theexemplary embodiments thereof, those skilled in the art will be able tomake various modifications to the described embodiments of the inventionwithout departing from the true spirit and scope of the invention. Theterms and descriptions used herein are set forth by way of illustrationonly and are not meant as limitations. In particular, although themethod of the present invention has been described by examples, thesteps of the method may be performed in a different order thanillustrated or simultaneously. Those skilled in the art will recognizethat these other variations are possible within the spirit and scope ofthe invention as defined in the following claims and their equivalents.APPENDIX A Number Revision Specification A.1 Reference Set 1 FF-801 FS1.4 Network Management FF-806 FS 1.0 Data Link Protocol - BridgeOperation Addendum FF-821 FS 1.4 Data Link Services Subset FF-822 FS 1.4Data Link Protocol Subset FF-870 FS 1.4 Fieldbus Message SpecificationFF-875 FS 1.4 Fieldbus Access Sublayer FF-880 FS 1.4 System ManagementFF-890 FS 1.4 Function Block Application Process-Part 1 A.2 ReferenceSet 2 FF-803 FS 1.0 HSE Network Management FF-586 FS 1.0 HSE EthernetPresence FF-588 FS 1.0 HSE Field Device Access Agent FF-589 FS 1.0 HSESystem Management FF-593 PS 2.0 HSE Redundancy

APPENDIX B RFC Number RFC Title 768 User Datagram Protocol (UDP) 791Internet Protocol (IP) Amended by:  IP Subnet Extensions, RFC 950  IPBroadcast Datagrams, RFC 919  IP Broadcast Datagrams with Subnets, RFC922 792 Internet Control Message Protocol (ICMP) 793 Transport ControlProtocol (TCP) 826 Ethernet Address Resolution Protocol (ARP) 894Ethernet Framing of IP Datagrams over Ethernet 1042 IEEE 802.2&3 Framingof IP Datagrams over Ethernet 1112 Internet Group Management Protocol(IGMP) 1122 Requirements for Internet Hosts - Communication Layers 1155Structure and Identification of Management Information 1157 SimpleNetwork Management Protocol (SNMP) 1213 Management Information Base-II(MIB II) 1533 DHCP Option and BOOTP Vendor Extensions 1541 Dynamic HostConfiguration Protocol (DHCP) 1643 Definitions of Managed Objects forthe Ethernet-like Interface Types 2030 Simple Network Time Protocol(SNTP)

1. An apparatus in a distributed control system, comprising: a firstnetwork interface for communicating with a first network having acommunication protocol stack; and a device access agent for mapping atleast one legacy format service message of said distributed controlsystem to a network format messages compatible with said communicationprotocol stack.
 2. The apparatus according to claim 1, wherein: saidfirst network comprising a commercial off-the-shelf Ethernet network. 3.The apparatus according to claim 2, further comprising: a high speedEthernet management agent for managing transport control protocol, userdata protocol, and Internet protocol layers of said communicationprotocol stack; and a high speed Ethernet management agent interfacethrough which said device access agent communicates with said high speedEthernet management agent.
 4. The apparatus according to claim 3,further comprising: a user data protocol local interface through whichsaid device access agent communicates with a user data protocol layer ofsaid communication protocol stack.
 5. The apparatus according to claim4, further comprising: a transport control protocol local interfacethrough which said device access agent communicates with a transportcontrol protocol layer of said communication protocol stack.
 6. Theapparatus according to claim 5, wherein said high speed Ethernetmanagement comprises: a management information base constructed with astandardized structure to thereby allow an open and interoperableprofile, and to make said apparatus to appear as a well behaved node. 7.The apparatus according to claim 1, further comprising: a networkmanagement information base for storing information necessary formanaging operation of said distributed control system; and a networkmanagement information base local interface through which said deviceaccess agent communicates with said network management information base.8. The apparatus according to claim 7, further comprising: a systemmanagement information base for storing system configuration informationof said apparatus; and a system management information base localinterface through which said device access agent communicates with saidsystem management information base.
 9. The apparatus according to claim8, further comprising: a system management kernel for configuring saidapparatus and storing system configuration information in said systemmanagement information base; and a system management kernel localinterface through which said device access agent communicates with saidsystem management kernel.
 10. The apparatus according to claim 9,further comprising: a local time clock for providing a local time foruse within said apparatus; and a system time clock for providing asystem time across said distributed control system; wherein said systemmanagement kernel synchronizes said local time clock with said a systemtime clock.
 11. The apparatus according to claim 10, further comprising:a redundancy entity for sending and receiving diagnostic informationover said first network; and a redundancy entity local interface throughwhich said device access agent communicates with said redundancy entity.12. The apparatus according to claim 11, wherein: said first networkinterface comprises a redundant plurality of first network interfaces;wherein said first network comprises a redundant plurality of firstnetworks; and wherein said redundancy entity maintains a network statustable indicating diagnostic status of said distributed control system toselect operational one of said redundant plurality of first networkinterfaces based on said network status table.
 13. The apparatusaccording to claim 10, further comprising: at least one function blockapplication process virtual field device for providing standardizeddefinitions of inputs, outputs, algorithms, control variables, andbehavior of said distributed control system; and at least one functionblock application process virtual field device interface through whichsaid device access agent communicates with said at least one functionblock application process virtual field device.
 14. The apparatusaccording to claim 1, further comprising: a second network interface forcommunicating with a second network using said at least one legacyformat service message.
 15. The apparatus according to claim 14, whereinsaid second network interface comprises: a plurality of second networkinterfaces.
 16. An open interoperable apparatus in a distributed controlsystem, comprising: a local time clock for providing a local time foruse within said apparatus; a system time clock for providing a systemtime across said distributed control system; and a system managementkernel for synchronizing said local time clock with said system timeclock.
 17. The open interoperable apparatus in accordance with claim 16,further comprising: a first network interface for communicating with afirst network having a communication protocol stack; a device accessagent for mapping at least one legacy format service message of saiddistributed control system to a network format messages compatible withsaid communication protocol stack; and a system management kernel localinterface through which said device access agent communicates with saidsystem management kernel.
 18. An open interoperable apparatus in adistributed control system, comprising: a redundant plurality of firstnetwork interfaces for communicating with respective ones of a redundantplurality of first networks having a communication protocol stack; and aredundancy entity configured to send and receive diagnostic informationthrough said redundant plurality of first network interfaces, saidredundancy entity maintaining a network status table indicatingdiagnostic status of said redundant plurality of first networks, andsaid redundancy entity being configured to select an operational one ofsaid redundant plurality of first networks based on said network statustable.
 19. The open interoperable apparatus according to claim 18,further comprising: a device access agent for mapping at least onelegacy format service message of said distributed control system to anetwork format messages compatible with said communication protocolstack; and a redundancy entity local interface through which said deviceaccess agent communicates with said redundancy entity.
 20. The openinteroperable apparatus according to claim 19, further comprising: asystem management kernel for configuring said apparatus andsynchronizing a local time clock, which provides a local time for usewithin said apparatus, with a system time clock, which provides a systemtime across said distributed control system; and a system managementkernel local interface through which said device access agentcommunicates with said system management kernel.
 21. An openinteroperable distributed control system, comprising: at least one firstnetwork having a communication protocol stack; and at least one devicein communication with said at least one first network, said at least onedevice having an access agent for mapping at least one legacy formatservice message of said open interoperable distributed control system toa network format message compatible with said communication protocolstack.
 22. The open interoperable distributed control system accordingto claim 21, wherein: said at lease one first network comprises acommercial off-the-shelf Ethernet network.
 23. The open interoperabledistributed control system according to claim 21, wherein said at leastone device further comprises: a system management kernel for configuringsaid apparatus and synchronizing a local time clock, which provides alocal time for use within said apparatus, with a system time clock,which provides a system time across said distributed control system; anda system management kernel local interface through which said deviceaccess agent communicates with said system management kernel.
 24. Theopen interoperable distributed control system according to claim 21,wherein: said at least one first network comprises a redundant pluralityof first networks; and wherein said at least one device furthercomprises: a redundancy entity configured to send and receive diagnosticinformation to and from said redundant plurality of first networks, saidredundancy entity maintaining a network status table indicatingdiagnostic status of said redundant plurality of first networks, andsaid redundancy entity being configured to select an operational one ofsaid redundant plurality of first networks based on said network statustable.
 25. The open interoperable distributed control system accordingto claim 24, wherein: said redundant plurality of first networkscomprises a redundant plurality of commercial off-the-shelf Ethernetnetworks.
 26. The open interoperable distributed control systemaccording to claim 21, further comprising: a plurality of secondnetworks, each of said plurality of second networks using said at leastone legacy service message format; wherein said at least one devicecomprises a redundant plurality of devices, each of said redundantplurality of devices comprises: a plurality of second network interfacesfor communicating with said plurality of second networks; and aredundancy entity configured to provide information necessary forselection of an operational one of said redundant plurality of devicesbased on a network status table indicating diagnostic status of at leastone of said redundant plurality of devices and said at least one firstnetwork.
 27. A method of synchronizing a plurality of device specificlocal times and a system time in an open interoperable distributedcontrol system, said plurality of device specific local times beingassociated with respective ones of devices in said open interoperabledistributed control system, said method comprising: detecting an end ofprevious operational cycle; providing a start time of a next operationalcycle to each of said plurality of devices; computing an offset betweeneach of said plurality of device specific local times and said systemtime; synchronizing each of said plurality of device specific localtimes with said system time using said computed offset; and aligningsaid plurality of device specific local times with respect to each otherso that said start time of said plurality of devices coincide.
 28. Themethod of synchronizing a plurality of device specific local times and asystem time in accordance with claim 27, further comprising: providing atime master in said open interoperable distributed control system, saidtime master maintaining a global time; determining whether said systemtime is synchronized with said global time; and setting a synchronizedflag if it is determined that said system time is synchronized with saidglobal time.
 29. The method of synchronizing a plurality of devicespecific local times and a system time in accordance with claim 27,wherein said step of aligning said plurality of device specific localtimes comprises: computing an offset between each of said plurality ofdevice specific local times with respect to each other; and adding atime delay to at least one of said plurality of device so that saidstart time of each of said plurality of devices coincide with respect toeach other.